Performance modeling of hierarchical memories

As the modern computing environment expands memory hierarchy from CPU registers and local memory to network storage, the optimal goal of a computer architect becomes to design a memory hierarchy that maximizes the overall design of his machine with a minimal cost. This requires deciding on the number, speed and size of the hierarchical layers. As the gap between processor and memory speed is growing exponentially, it becomes more important to develop an analytical model to capture all these hierarchical levels and optimize the memory access time to make a good utilization of both CPU and memory. In this paper we study the performance of systems with multi-level hierarchical memories by modeling their access time which helps the designer optimize the cost and access time. We use a linear-algebraic queuing theory approach to achieve our goal and we explain why previous attempts failed to provide accurate models. Our model differs from all the previous related work by being global and general and by using some probabilistic equations that show the interdependence between the different levels and by using the P-K formula to distinguish between the memory access time and queuing time. Our approach is independent of the application using the memory while classical approaches were program dependent. Moreover, our model achieves higher levels of accuracy while being expandable to multiple levels.